feat: Codex voice conversations on web, desktop, and mobile by jakeleventhal · Pull Request #13168 · pingdotgg/t3code

jakeleventhal · 2026-09-23T00:20:33Z

Screen.Recording.2026-09-22.at.8.13.50.PM.mov

ScreenRecording_09-22-2026.20-15-53_1.MP4

Codex 0.156 shipped voice conversations: you talk to a realtime voice model, it hands coding work to Codex in the thread, and it reads Codex's answer back. T3 Code had no way to do this.

How it works

Audio stays off the T3 socket. The client owns the mic, speaker, and WebRTC peer (oai-events data channel), and audio flows directly between the client and OpenAI. The new provider.voice.session stream RPC carries only the SDP handshake, captions, and lifecycle, so voice costs the same locally, over a relay, or through a tunnel.
The stream is the conversation. Unsubscribing or disconnecting stops the Codex realtime session, so a dropped client can't leave a billed session running.
Server (CodexVoice.ts, CodexAdapterV2). Starts thread/realtime/start with the same V3 WebRTC, client-managed-handoff settings the Codex TUI uses. When a voice turn finishes, its final answer goes back through thread/realtime/appendSpeech; commentary and reasoning are never spoken.
Idle-thread handoffs. Codex starts or steers the delegated turn itself, and that can't be turned off. Steering an active run needs nothing special. On an idle thread V2 has no provider-initiated run, so the adapter interrupts Codex's turn and, once it settles, offers a voice continuation. The request then runs as an ordinary user message. The tradeoff: Codex's own history keeps a stub of the interrupted turn. First-class provider-initiated runs would remove this.
Clients. A shared VoiceModeController in client-runtime handles negotiation, mute, captions, mic/speaker levels, the connect timeout, and teardown. Browsers and react-native-webrtc share the same structural peer API.
Web/desktop. Waveform button in the composer, /voice, F8 (voice.toggle), an unbound voice.toggleMute, command palette actions, and a voice strip with status, meters, mute/stop, and live captions. Captions are ephemeral.
Desktop. Grants mic access to the app origin only, asks macOS on first use, and adds NSMicrophoneUsageDescription and the audio-input entitlement.
Mobile. Adds react-native-webrtc (native rebuild required), Android RECORD_AUDIO, loudspeaker routing, and the same button and strip. Conversations stop on background or when leaving the thread.
Settings and scope. Voice is chosen in Codex provider settings. It's Codex-only (ServerProvider.supportsVoice) and needs a thread whose provider session is running.

Verification

Typecheck is clean in contracts, shared, client-runtime, server, web, desktop, and mobile.
Focused tests: 188 server (voice module, V2 Codex adapter, continuation service), 7 client-runtime, 88 web, 74 desktop, 69 shared keybindings, and 5 mobile.
Codex 0.156's app-server accepts the exact start params.
Installed and launched on desktop and iPhone from a personal build. A full spoken round trip has not been verified yet, and no screenshots are attached.

🤖 Generated with Claude Code (Claude Opus 5.5, running in T3 Code)

macroscopeapp · 2026-09-23T00:24:46Z

Approvability

Verdict: Not approved

Macroscope's review found this PR not approvable — This PR adds a substantial cross-platform Codex voice workflow with WebRTC, microphone permissions, native entitlements, new server orchestration, and an authenticated streaming RPC. It also changes the default F8 shortcut and adds static-analysis suppressions, so the scope and side effects warrant human review.

^{You can add or adjust custom eligibility rules. Learn more.}

macroscopeapp · 2026-09-23T00:36:52Z

Ignore my attempted follow-up review submission; it was not posted. The submitted inline comment on ChatComposer.tsx is the only UI consistency finding from this check.

^{Posted via Macroscope — UI Consistency}

…sults

…gg#13462) Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

…ingdotgg#13467) Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

…gg#13476) Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>

…#13470) Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

…g#13471) Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

…oval (pingdotgg#13486) Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

…fallback (pingdotgg#13494) Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

…base (pingdotgg#13502) Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

…gg#13499) Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

…ingdotgg#13493) Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

…13509) Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

pingdotgg#13505) Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

…ingdotgg#13508) Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

Codex 0.156 voice mode for T3 Code: talk to a realtime voice model that hands work to Codex in the thread and reads its answers back. Audio runs over WebRTC between the client and OpenAI; the server relays only SDP and captions over provider.voice.session, and ending the stream stops the conversation. On orchestrator V2, Codex's self-started voice turns are interrupted and resubmitted as ordinary user runs, since V2 has no provider-initiated runs. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

- Resubmit a Codex-started voice turn only once it reports interrupted, so a turn that finished before the interrupt landed never runs twice. - Carry each pending resubmit's own provider thread, and keep a replaced voice stream from clearing its successor's thread. - Mobile: gate voice on the provider thread existing, keep the voice strip reachable while a question card hides the composer, and keep a newer conversation's wake lock. - Drop unused exports flagged by knip and update the Codex settings fields test. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

…versation - Resubmit a Codex-started voice turn that failed, not only one that was interrupted, so the spoken request is never silently dropped. - /voice stops an active conversation even if the provider no longer reports voice support. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

…ional shortcut Startup default sync only treated identical `when` strings as conflicts, so a default like `f8 voice.toggle when !terminalFocus` was appended after a user's bare `f8` binding and won whenever the terminal lacked focus. A default is now skipped when its shortcut matches a user rule and either side is unconditional. Also corrects the voice lifecycle comment: screens pushed over the thread keep a conversation running. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

juliusmarminge · 2026-10-01T18:35:53Z

Note

This comment is posted by Julius' dot

Closing under the prior-approval rule. This adds a new per-thread live voice workflow across web, desktop, and mobile, including voice-to-Codex handoffs. The recordings and focused checks demonstrate the proposal, but no maintainer approval of its direction and scope is linked. Please agree on that scope with maintainers, link their approval here, and request reconsideration.

github-actions Bot added vouch:trusted PR author is trusted by repo permissions or the VOUCHED list. size:XXL 1,000+ changed lines (additions + deletions). labels Sep 23, 2026

macroscopeapp Bot reviewed Sep 23, 2026

View reviewed changes

Comment thread apps/web/src/components/chat/ChatComposer.tsx Outdated

macroscopeapp Bot reviewed Sep 23, 2026

View reviewed changes

Comment thread packages/contracts/src/provider.ts

Comment thread apps/server/src/provider/Layers/CodexVoice.ts Outdated

macroscopeapp Bot reviewed Sep 23, 2026

View reviewed changes

Comment thread apps/server/src/provider/Layers/CodexVoice.ts

juliusmarminge force-pushed the t3code/codex-turn-mapping branch from a419382 to f2bdd9f Compare September 23, 2026 00:50

jakeleventhal force-pushed the t3code/codex-voice-mode branch from 2ace4c3 to b2614fe Compare September 23, 2026 00:59

macroscopeapp Bot reviewed Sep 23, 2026

View reviewed changes

Comment thread apps/server/src/orchestration-v2/Adapters/CodexAdapterV2.ts Outdated

Comment thread apps/web/src/components/ChatView.tsx Outdated

juliusmarminge force-pushed the t3code/codex-turn-mapping branch 5 times, most recently from 6ca6a24 to 3e4ca4c Compare September 23, 2026 23:24

saphid mentioned this pull request Sep 24, 2026

feat(voice): GPT Live voice agent on Orchestrator V2 #13214

Closed

3 of 4 tasks

juliusmarminge force-pushed the t3code/codex-turn-mapping branch 2 times, most recently from 9fc8676 to ae065ff Compare September 24, 2026 21:34

juliusmarminge added 11 commits September 24, 2026 16:11

fix(server): wait for nested completion delivery before publishing re…

235cab6

…sults

test(server): verify background delivery with real providers

8db9174

fix: keep working timers anchored to the active run

c7e79df

perf: page complete turns and bound timeline reconciliation

39732de

perf(client): narrow thread subscriptions and navigation updates

5869eca

feat(mobile): manage queued messages in a dedicated sheet

d4c007b

fix(web): fold completed trailing background activity

bd9906b

fix: reconcile main settings and previews with orchestration v2

0d8c0d2

perf(client): reconcile replay batching with orchestration v2

0ae90ed

fix: reconcile main Codex model selection and UI updates

8e42403

fix: reconcile main context previews and rewind updates

79e534a

juliusmarminge and others added 15 commits September 24, 2026 16:12

refactor(server): remove the unused V1 Codex session runtime (pingdot…

2393dd3

…gg#13462) Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

fix(server): Codex V2 sessions use the configured launch arguments (p…

426a83a

…ingdotgg#13467) Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

fix(server): merged PRs update and auto-settle threads again (pingdot…

9b73af2

…gg#13476) Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>

fix(server): Codex V2 sessions expand ~ in the binary path (pingdotgg…

7ca9e0a

…#13470) Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

fix(server): Claude V2 sessions expand ~ in the binary path (pingdotg…

ce0c09b

…g#13471) Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

chore(server): unexport OpenCode capabilities left unused by test rem…

21bef44

…oval (pingdotgg#13486) Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

test(server): make the provider-switch resume-failure test reach the …

288d4bc

…fallback (pingdotgg#13494) Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

fix(v2): reconcile latest main protocol and theme updates

2f12ebf

chore(web): drop the right panel sheet class left unused by the V2 re…

6ebb927

…base (pingdotgg#13502) Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

fix(server): Cursor V2 threads load project skills and rules (pingdot…

38fd911

…gg#13499) Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

test(server): re-record Cursor replay fixtures and cover skills live (p…

7384376

…ingdotgg#13493) Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

fix(codex): replay legacy user-input requests as blocking (pingdotgg#…

53880fd

…13509) Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

test(server): record the Codex thread rollback fixture live on 0.156.1 (

1c60bcf

pingdotgg#13505) Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

test(server): record Codex multi-agent V2 subagents live on 0.156.1 (p…

0c0a606

…ingdotgg#13508) Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

fix(v2): reconcile main auto-settle controls and preview migrations

1563a64

juliusmarminge force-pushed the t3code/codex-turn-mapping branch from 6448cfe to 1563a64 Compare September 24, 2026 23:21

jakeleventhal and others added 5 commits September 24, 2026 19:37

refactor: use Toggle for the voice button and a schema voice error

63f2a69

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

fix(web): dim the disabled voice glyph instead of restyling Toggle

5d37d88

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

jakeleventhal force-pushed the t3code/codex-voice-mode branch from 46603d5 to 9d8c964 Compare September 24, 2026 23:42

macroscopeapp Bot reviewed Sep 24, 2026

View reviewed changes

Comment thread packages/shared/src/keybindings.ts

macroscopeapp Bot reviewed Sep 24, 2026

View reviewed changes

Comment thread apps/server/src/ws.ts

juliusmarminge force-pushed the t3code/codex-turn-mapping branch from fe4f6ad to 87c67bd Compare September 25, 2026 05:55

juliusmarminge closed this Oct 1, 2026

This was referenced Oct 1, 2026

feat: Codex voice conversations on web, desktop, and mobile #14638

Closed

feat: Codex voice conversations on web, desktop, and mobile #14865

Open

Navigation Menu

Repository navigation

feat: Codex voice conversations on web, desktop, and mobile - #13168

feat: Codex voice conversations on web, desktop, and mobile#13168
jakeleventhal wants to merge 678 commits into
pingdotgg:t3code/codex-turn-mappingfrom
jakeleventhal:t3code/codex-voice-mode

jakeleventhal commented Sep 23, 2026 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

macroscopeapp Bot commented Sep 23, 2026 •

edited

Loading

Uh oh!

Uh oh!

macroscopeapp Bot commented Sep 23, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

juliusmarminge commented Oct 1, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

15 participants

Conversation

jakeleventhal commented Sep 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

How it works

Verification

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

macroscopeapp Bot commented Sep 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Approvability

Uh oh!

Uh oh!

macroscopeapp Bot commented Sep 23, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

juliusmarminge commented Oct 1, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

15 participants

jakeleventhal commented Sep 23, 2026 •

edited

Loading

macroscopeapp Bot commented Sep 23, 2026 •

edited

Loading